The real world has a lot more brightness variation than can be captured by the sensors available in most cameras today. The radiance of a single scene may contain four orders of magnitude in brightness from shadows to fully lit regions. Typical CCD or CMOS sensors only capture about 256-1024 brightness levels.
This limited dynamic range problem has inspired many solutions in recent years. One method of capturing the full radiance of a static scene is to take multiple still exposures of the scene and then to combine them to create a High Dynamic Range (HDR) map. Dynamic range shows the ratio of a specified maximum level of a parameter to the minimum detectable value of that parameter. High dynamic range imaging allows a greater dynamic range of exposures than normal digital imaging techniques. For example, a high dynamic range image can accurately represent the wide range of intensity levels found in real scenes, ranging from direct sunlight to the dark shadows. In a normal digital imaging technique, either the dark region or the bright region becomes saturated, or almost saturated, so that the details in these regions become unrecognizable.
Typically, a digital camera has a limited dynamic range. With a given exposure setting, the digital camera may not be able to capture the details in a bright area in the scene since the bright area in the picture taken by the digital camera is saturated and represented as a uniform white region. Similarly, the details in a dark area in the scene may be captured as a uniform black region or a region where noise dominates the scene content. Increasing the exposure duration may allow the camera to capture more details in the dark region but lose more details near the bright region as the bright region expands. Reducing the exposure duration may allow the camera to capture more details in the bright region but lose more details near the dark region as the dark region expands.
Paul E. Debevec and Jitendar Malik (Recovering high dynamic range radiance maps from photographs, Proc. of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, pp. 369-378, 1997) present a method to derive a response curve that relates the pixel image values and the natural logarithm of the product of irradiance and exposure duration. The response curve then can be used to recover high dynamic range radiance maps from series of images of a same scene taken at different exposure levels. In one example. Paul E. Debevec and Jitendar Malik map the logarithm of the radiance values into a gray scale image, which presents all the information in the series of images taken at the different exposure levels.
To obtain the response curve, Paul E. Debevec and Jitendar Malik formulated a quadratic objective function for least square minimization. The minimization process is formulated to solve for both the response curve and for the irradiances involved simultaneously. The quadratic objective function involves the second derivative of the response curve in order to obtain a smooth response curve.
Methods and apparatuses to derive at least one exposure function for high dynamic range imaging from images of different exposure durations are described by Marcu et al in U.S. Pat. No. 7,626,614. A method to generate a high dynamic range image from a plurality of images taken with a plurality of exposure durations described in U.S. Pat. No. 7,142,723 B2 by Kang et al includes: computing an exposure function from the plurality of images; and combining the plurality of images into a high dynamic range (HDR) image using the exposure function. A HDR image may be considered to be an image having a range of luminance (or other color intensity value or a combination of such values) which is greater than the average range of a plurality of images (and usually the HDR image has a range which is greater than the range of any image in the plurality of images which was used to created the HDR image). Both these method will calculate scene illuminance based upon known exposure durations.
U.S. Pat. No. 6,040,858 by Ikeda discloses the composite way by an image processing method of expanding the dynamic range by determining a saturated or noise region in an image signal and replacing the image signal with a proper image signal. A threshold value is set in units of colors on the basis of a standard image signal for each color, which is obtained upon sensing an image in a standard exposure and a non-standard image signal for each color, which is obtained upon sensing the image in a non-standard exposure. A saturated or noise region is determined by applying the threshold value set in units of colors to the standard image signal. A region determined as a saturated or noise region is replaced with the non-standard image signal.
The dynamic range of the captured image is limited not only by the image sensor. Optical flare also reduces the practical dynamic range of an image as well. Consider a lens with 1% veiling glare metric. Assuming an average scene reflectance of 18% implies that 0.18% of flare will superimposed upon the image. This effectively limits the amount of shadow detail or limits the dynamic range of an image to about 9 stops.
Methods to increase the dynamic range of images acquired by an image sensing device would allow such images to be rebalanced to achieve a more pleasing rendition of the image. Also, images with increased dynamic range would allow for more pleasing contrast improvements, such as described by Lee et al. in commonly assigned U.S. Pat. No. 5,012,333.
One method used for obtaining improved images with an image sensing device is exposure bracketing, whereby multiple images are captured at a range of different exposures, and one of the images is selected as a best overall exposure. This technique, however, does not increase the dynamic range of any individual image captured by the image sensing device.
Of the methods described, none incorporate an estimate of image flare that can dominate shadows and obscure image details in those regions.
Therefore, a need in the art exists for an improved solution to combining multiple images to form an image having increased dynamic range, without requiring special hardware or additional image sensors, without sacrificing performance for scenes not requiring high dynamic range, without requiring a knowledge of the exposure, and with dynamic range limitation caused by flare.